The invention will be described, for illustration, in the context of a robotic arc-welding seam tracking system. It involves two aspects: first, the automatic guidance of the robot, or other manipulator of an effector end, to follow the seam path as sensed ahead of the tool, and secondly, image processing conceived and implemented for the detection of the seam path and for the generation of parametric information to be used in controlling the industrial robot.
It is known from "Progress In Visual Feedback For Robot Arc-Welding Of Thin Sheet Steel" by W. F. Clocksin, P. G. Davey, C. G. Morgan and A. R. Vidler, in Proceedings of the 2nd Int. Conf. on Robot Vision and Sensory Control, 1982, pp. 189-200:
(1) to generate with a laser beam a light stripe on the seam to be tracked, and to derive with a camera a stripe image in a two-coordinate plane, then, to use triangulation to represent in a three-coordinate system a recognizable representation of the joint to be welded;
(2) to compute and store digitally in a data base the information associated with each point identified by a stripe along the seam;
(3) to use optically derived data to make a model defining robot positioning and to derive between triangulated optically and digitally derived information errors regarding the actual position of the gap of the joint, the root corner of the joint, and any standoff as well as lateral errors; and
(4) to use such errors for correction in controlling a welding torch.
However, in the afore-stated reference, optical imaging and joint recognition are effected separately from torch control in the course of two successive seam paths, the second only being used for effective welding.
It is known from "Adaptive Robotic Welding Using A Rapid Image Pre-Processor" by M. Dufour and G. Begin, in Robot Vision and Sensory Controls, 3rd Conf. Cambridge, Mass., 6-10 November 1983, pp. 641-648 to effectuate visual sensing and welding simultaneously, while accomplishing adaptive robotic welding. Real time control of the robot is the object. However, the article contemplates feedback control of the robot in response to the information obtained by optical sensing.
It is known, from "Tracking Control System For Arc Welding Using Image Sensor" by M. Kawahara and H. Matsui in Proceedings of the Eighth Triennial World Congress of the Intern. Fed. of Automatic Control, Kyoto, Japan, 24-28 Aug. 1981 (Oxford, England; Pergamon Press 1982-pp. 2117-2122 vol. 4), to use data derived from a seam stripe image for joint recognition and to compute the groove center position from which to establish groove pattern recognition and torch positioning control. Optical detection is made ahead of the torch, and the target value obtained from such optical data is delayed at each particular point, to be applied for control at a time the torch reaches the optically sensed point.
This prior art, however, assumes a simple correlation between the two displaced locations, for sensing and welding, respectively, not encountered in practice with an industrial robot, where six degrees of freedom and a relatively complex seam are to be operated on.
It is known, from "A Visual Sensor For ArcWelding Robots" by Takao Bamba, Hisaichi Maruyama, Eiichi Ohuo and Yashunori Shiga in Proceedings, 11th Intern. Symposium on Industrial Robots, Tokyo, Japan, 1981, pp. 151-158:
(1) to pass a slit beam of light upon the track to be followed so as to derive a series of stripes;
(2) to derive from a stripe by triangulation two-dimensional image of the joint;
(3) to detect an horizontal deviation of the robot position from the center in the sensing plane;
(4) to feed to a robot control system the deviation signal for correction of the torch path.
In this prior art, control of the robot is not analyzed and no other solution is proposed than under conventional feedback operation.
It is known from: "A Visual Seam Tracking System for Arc-Welding Robots" by T. Bamba, H. Muruyama, N. Kodaira and E. Tsuda presented at the 14th Intern. Symposium on Industrial Robots, Gothenburg, Sweden, 2-4 October 1984, pp. 365-373, to provide visual guidance in controlling an arc-welding robot having five degrees of freedom. Sensing, however, is effected by scanning the joint area cyclically while rotating the sensing unit about the welding torch, and control of the robot is implemented directly without consideration of feedforward.
It is known from, "Model Driven Vision To Control A Surface Finishing Robot" by D. Graham, S. A. Jenkins and J. R. Woodwark presented at Robot Vision and Sensory Controls, 3rd Conf. Cambridge, Mass., 6-10 November 1983, pp. 433-439, to control a robot under six degrees of freedom in response to a camera image treated and interpreted by a computer. There, however, the problem of real time control has been solved by the use of off-line analysis and the determination of preplanned paths, as well as of computer models, for the determination of the robot kinetics.
It is known from "Data Processing Problems For Gas Metal Arc (GMA) Welder" by G. Nachev, B. Petkov, and L. Blagoev presented at Robot Vision and Sensory Controls, 3rd Conf. Cambridge, Mass., 6-10 November 1983, pp. 675-680, to detect five characteristic points of the weld joint to be used in an optical sensor coordinate system and to provide therefrom information using coordinate transformations for control of an industrial robot. This approach, however, assumes an adaptation of the robot to such implementation rather than providing control commands externally from the robot inherent characteristics.
It is generally known, from "A Real-Time Optical Profile Sensor For Robot Arc Welding" by G. L. Oomen and W. J. P. A. Verbeek, presented at Robot Vision and Sensory Controls, 3rd Conf. Cambridge, Mass., 6-10 November, pp. 659-668, to derive from an optical sensing system coordinates of the path to be followed by the torch, using transformation from camera coordinates to workpiece coordinates to place with a robot controller the torch in correct position. How this is done has not be shown, and the problems there involved were not considered, nor solutions were given.
It is generally known from "Joint Tracking and Adaptive Robotic Welding Using Vision Sensing of the Weld Joint Geometry" by J. E. Agapakis, J. M. Katz, M. Koifman, G. N. Epstein, J. M. Friedman, D. O. Eyring and H. J. Rutishauser in Welding Journal, November 1986, pp. 33-41 to use vision processing with stripe extraction to recognize the significant features of a weld joint for visual guidance of a welding robot moving under a taught path, and by interpolation to locate the joint root and its distance from the taught path for robot control and correction. This technique, however, makes use of the taught path of the robot, and it fails to implement a feedforward approach.
Different look-ahead techniques have been used, in order to project the next move of an effector end while computing and controlling in anticipation of where the effector end shall be. See for instance: U.S. Pat. Nos. 4,501,950; 4,542,279; 4,590,356; 4,663,726; and 4,675,502. However, none of these is teaching how to correlate in real time and under feedforward control the sensed locations and the extrapolated taught path location in order to determine the required compensation in the inherent robot manipulations along the sensed seam, or joint. For instance, U.S. Pat. Nos. 4,590,356; 4,542,279 and 4,501,950 essentially involve terminal homing, thus no static taught path is controlling. U.S. Pat. No. 4,663,726 is concerned with the robot system itself, i.e. the robot is not an a priori for the automatic seam tracker.
U.S. Pat. No. 4,675,502 involves a real time controller for a robot utilizing the robot taught path with a look-ahead vision system. The converted sensed locations data are stored in a queue, and a slightly ahead of the tool location is retrieved as a target, which is compared with the taught path for correction and control of the tool movement from present. No taught path recovery is taking place, and the elapsed distance, rather than the velocity, is taken into consideration for establishing under feedforward control the anticipated tool position to be aimed at with the robot.
The invention from one aspect thereof uses an interpolation method. In this regard, the prior art shows with U.S. Pat. No. 4,683,543 time-based interpolation control of a robot. However, in contrast with the present invention, it is not there to interpolate on the basis of elapsed distances. Another such time-based interpolation technique is shown in U.S. Pat. No. 4,663,726.